Red Hat Enterprise Linux Implementation

Remote Block Storage

Module Topics

Introduction to iSCSI
iSCSI Fundamentals
iSCSI Target Overview
iSCSI Target Configuration
Authentication
Command-Line Mode
iSCSI Initiator Introduction

Introduction to iSCSI

Internet Small Computer System Interface (iSCSI)

TCP/IP-based protocol
- Emulates SCSI high-performance local storage bus over IP networks
- Provides data transfer and management to remote block storage devices
Storage area network (SAN) protocol
- Extends SANs across local/wide area networks
- Provides location-independent data storage retrieval

block_storage_scsi_topologies

Introduction to iSCSI

SCSI protocol suite provides Command Descriptor Block (CDB) command set over device bus communication protocol
Original SCSI topology used cabled bus
- Physical cabling with 20-meter limitation for all devices per channel
Devices used unique numeric target IDs
- 0-7 or 0-15 with dual channel
Fibre Channel (FC) obsoleted physical SCSI disks and cabling
- Retains SCSI CDB command set
- Replaces disk and bus communication with protocols for optical cabling

Introduction to iSCSI

iSCSI protocol retains CDB command set
- Performs bus communication between iSCSI systems encapsulated over standard TCP/IP
iSCSI servers emulate SCSI devices using files, logical volumes, disks of any type as underlying storage (backstore)
- Presented as targets
iSCSI service typically implemented in software above operating system TCP/IP stack or TCP offload engine (TOE)
- TOE: Specialized Ethernet network interface card (NIC) that includes TCP/IP network layers to increase performance
Can also hardware-implement iSCSI as host bus adapter (HBA) for greater performance increase

Introduction to iSCSI

Storage Area Network (SAN) Traffic

Enterprise SANs require dedicated traffic infrastructure
FC’s independent optical cabling and switches guarantee isolation
Implement iSCSI on cabling independent of standard LAN traffic
- Performance can degrade with bandwidth congestion on shared networks
Ethernet and FC offer copper and optical cabling options
- Allows network consolidation with traffic classification
SAN traffic typically unencrypted
- Physical cabling normally enclosed within secure data centers
iSCSI and FCoE can use IPSec
Preferred NICs, TOEs, HBAs can provide encryption
iSCSI offers CHAP usernames and passwords for authentication
- Limits connectivity between initiators and targets

Transcript:

Enterprise-grade SANs require dedicated traffic infrastructure. FC’s independent optical cabling and switches guarantees isolation. iSCSI should also be implemented on cabling that is independent of standard LAN traffic, since performance can degrade due to bandwidth congestion on shared networks. Both Ethernet and FC now offer copper and optical cabling options, allowing network consolidation combined with traffic classification.

Storage area network traffic is typically unencrypted, since physical server-to-storage cabling is normally enclosed within secure data centers. For WAN security, iSCSI and Fibre Channel over Ethernet (FCoE) can utilize Internet Protocol Security (IPSec), a protocol suite for securing IP network traffic. Select networking hardware (preferred NICs, TOEs, and HBAs) can provide encryption. iSCSI offers Challenge-Handshake Authentication Protocol (CHAP) usernames and passwords as an authentication mechanism to limit connectivity between chosen initiators and targets.

Introduction to iSCSI

Using iSCSI to Extend SANs

iSCSI previously not considered enterprise-grade storage option
- Ethernet: Slower 100 and 1000 Mb/s
- FC: 1 and 4 Gb/s optical infrastructure
Bandwidth availability now similar for both:
- Current:
  - Ethernet: 10 or 40 Gb/s
  - FC: 8, 10, 16, or 20 Gb/s
- Pending:
  - Ethernet: 100 Gb/s
  - FC: 32 or 128 Gb/s

block_storage_fc_topologies

Introduction to iSCSI

iSCSI extends SAN beyond the limits of local cabling
- Facilitates storage consolidation in local or remote data centers
iSCSI structures are logical
- New storage allocations made using only software configuration
- No need for cable or physical disks
Using multiple remote data centers, iSCSI simplifies:
- Data replication
- Migration
- Disaster recovery

iSCSI Fundamentals

iSCSI protocol functions in client/server configuration
Client systems configure initiator software
- Sends SCSI commands to remote server storage targets
Targets appear on client system as local, unformatted SCSI block devices
- Identical to devices connected with SCSI cabling,
  - FC direct attached, FC switched fabric

iSCSI Fundamentals

Term

Description

initiator

iSCSI client, typically available as software but also implemented as iSCSI HBAs
Must be given unique names (see IQN)

target

iSCSI storage resource, configured for connection from iSCSI server
Must be given unique names (see IQN)
Provides one or more numbered block devices called logical units (see LUN)
iSCSI server can provide many targets concurrently

ACL

Access Control List (entry): Access restriction using node IQN (commonly iSCSI Initiator Name) to validate access permissions for initiator

discovery

Queries target server to list configured targets
Target use requires additional access steps (see login)

IQN

iSCSI Qualified Name: Worldwide unique name to identify both initiators and targets, in mandated naming format: iqn.YYYY-MM.com.reversed.domain[:optional_string]
- iqn: Name will use domain as identifier
- YYYY-MM: First month domain name was owned
- com.reversed.domain: Reversed domain name of organization creating iSCSI name
- optional_string: Optional, colon-prefixed string assigned by domain owner as desired while remaining worldwide unique
  - May include colons to separate organization boundaries

login

Authenticating to target or LUN to begin client block device use

LUN

Logical Unit Number: Mumbered block devices attached to and available through target
One or more LUNs may be attached to single target
- Target typically provides only one LUN

node

iSCSI initiator or iSCSI target, identified by IQN

portal

IP address and port on target or initiator used to establish connections
Some iSCSI implementations use portal and node interchangeably

TPG

Target Portal Group: Set of interface IP addresses and TCP ports to which specific iSCSI target listens
Can add target configuration (e.g., ACLs) to TPG to coordinate settings for multiple LUNs

iSCSI Fundamentals

Access Control Lists (ACLs)

iSCSI uses ACLs to perform LUN masking
- Managing accessibility of targets and LUNs to initiators
Can also limimt access to targets CHAP authentication
iSCSI ACLs similar to FC device worldwide numbers (WWNs) for soft zoning management restrictions
FC switch-level compulsory port restriction (hard zoning) has no comparable iSCSI mechanism
- Ethernet VLANs could provide similar isolation security

iSCSI Fundamentals

Block Devices

iSCSI network-accessed block devices discoverable from remote initiators
Local file systems typically do not support concurrent multisystem mounting
- Can result in significant file system corruption
Clustered systems resolve multiple system access using Global File System (GFS2)
- Provides distributed file locking and concurrent multinode file system mounting

target_server_stack

iSCSI Fundamentals

Attached iSCSI block device appears as local SCSI block device (sdx)
- Use underneath local file system, swap space, or raw database installation
Contrast with use of network file server protocols
- Providing file I/O from remote file systems to local applications on multiple client systems concurrently

file_server_stack

References

iSCSI Target Overview

In SCSI terminology, target is single connectible storage/output device uniquely identified on SCSI bus
In iSCSI, target can be:
- Dedicated physical device in network-attached storage enclosure
- iSCSI software-configured logical device on networked storage server
Target is end point in SCSI bus communication
- Passes command descriptor blocks (CDB) to request or provide storage transactions

iSCSI Target Overview

Logical Unit Numbers (LUNs)

Target is configured with logical unit numbers (LUNs)
- Provide access to storage/output device
- Appear as the target’s sequentially numbered disk drives
- Targets typically have one LUN
Initiator performs SCSI negotiation with target to establish connection to LUN
LUN responds as emulated SCSI disk block device
- Can use in raw form or format with client-supported file system
Do not mount single-system file systems to more than one system at a time
iSCSI allows shared target and LUN access from multiple initiator nodes
- Requires use of cluster-capable file systems such as GFS2
Mounting file systems designed for local, single-system access from more than one system causes file system corruption

Transcript:

To provide access to the storage or output device, a target is configured with one or more logical unit numbers (LUNs). In iSCSI, LUNs appear as the target’s sequentially numbered disk drives, although targets typically have only one LUN. An initiator performs SCSI negotiation with a target to establish a connection to the LUN. The LUN responds as an emulated SCSI disk block device, which can be used in raw form or formatted with a client-supported file system.

Do not mount single-system file systems to more than one system at a time. iSCSI allows shared target and LUN access from multiple initiator nodes, requiring the use of cluster-capable file systems such as GFS2. Mounting standard file systems designed for local, single-system access (e.g., btrfs, ext3, ext4, FAT32, HPFS+, NTFS, XFS, ZFS) from more than one system concurrently will cause file system corruption.

iSCSI Target Overview

LUN Masking

iSCSI provides LUN masking
- Uses ACLs to restrict LUN accessibility to initiators
Unless shared access intended, ACLs ensure only designated client node can log in to target
On target server, can set ACLs:
- At TPG level to secure groups of LUNs
- Individually per LUN

iSCSI Target Configuration

Target Server Configuration Demonstration

targetcli both:
- Command-line utility
- Interactive shell to create, delete, configure iSCSI target components
Group target stack objects into hierarchical tree of objects
- Allows easy navigation and contextual configuration
Linux commands used in shell:
- cd
- ls
- pwd
- set
targetcli supports TAB completion to:
- Complete partially typed commands
- View acceptable keywords at location in command

iSCSI Target Configuration

Steps 1-2

Install targetcli if needed:

[root@server0 ~]# yum -y install targetcli

Run targetcli with no options:
```
[root@server0 ~]# targetcli
/> ls
```

iSCSI Target Configuration

Step 3

Create backing storage (backstores)

Type

Description

block

Block device defined on server
Disk drive, disk partition, logical volume, multipath device, any device files defined on server of type b

fileio

Create file of specified size, in server filesystem
Similar to using image files as storage for virtual machine disk images

pscsi

Physical SCSI
Permits passthrough to physical SCSI device connected to server
Not typically used

ramdisk

Create ramdisk device of specified size in server memory * Does not store data persistently * When server rebooted, ramdisk definition returns when target is instantiated, but all data is lost

Transcript:

iSCSI target configuration

Create backing storage (backstores). There are several types of backing storage.
- block - A block device defined on the server. A disk drive, disk partition, a logical volume, multipath device, any device files defined on the server that are of type b.
- fileio - Create a file, of a specified size, in the filesystem of the server. This method is similar to using image files to be the storage for virtual machine disk images.
- pscsi - Physical SCSI. Permits passthrough to a physical SCSI device connected to the server. This backstore type is not typically used.
- ramdisk - Create a ramdisk device, of a specified size, in memory on the server. This type of storage will not store data persistently. When the server is rebooted, the ramdisk definition will return when the target is instantiated, but all data will have been lost.

iSCSI Target Configuration

Example: Using existing logical volume, disk partition, and new file at specified size

Backstores display as deactivated:

/> cd /backstores/
/backstores> block/ create block1 /dev/iSCSI_vg/disk1_lv
Created block storage object block1 using /dev/iSCSI_vg/disk1_lv.
/backstores> block/ create block2 /dev/vdb2
Created block storage object block2 using /dev/vdb2.
/backstores> fileio/ create file1 /root/disk1_file 100M
Created fileio file1 with size 104857600
/backstores> ls
o- backstores ............................................................ [...]
  o- block ................................................ [Storage Objects: 2]
  | o- block1 ....... [/dev/iSCSI_vg/disk1_lv (100.0MiB) write-thru deactivated]
  | o- block2 ...................... [/dev/vdb2 (1.0GiB) write-thru deactivated]
  o- fileio ............................................... [Storage Objects: 1]
  | o- file1 .............. [/root/disk1_file (100.0MiB) write-back deactivated]
  o- pscsi ................................................ [Storage Objects: 0]
  o- ramdisk .............................................. [Storage Objects: 0]

iSCSI Target Configuration

Step 4

Create IQN for target:

Also creates default TPG underneath IQN

/backstores> cd /iscsi/
/iscsi> create iqn.2014-06.com.example:remotedisk1
Created target iqn.2014-06.com.example:remotedisk1.
Created TPG 1.
/iscsi> ls
o- iscsi .......................................................... [Targets: 1]
  o- iqn.2014-06.com.example:remotedisk1 ............................. [TPGs: 1]
    o- tpg1 ............................................. [no-gen-acls, no-auth]
      o- acls ........................................................ [ACLs: 0]
      o- luns ........................................................ [LUNs: 0]
      o- portals .................................................. [Portals: 0]

Can use create without specifying IQN value
- targetcli generates IQN similar to iqn.2003-01.org.linux-iscsi.server0.x8664:sn.69b30d2cfd01
- Specifying IQN value lets admin use meaningful namespace for IQN

iSCSI Target Configuration

Step 5

In TPG, create ACL for client node to use later:

Default: Global auto_add_mapped_luns parameter set to true

LUNs in TPG mapped to each ACL as it is created

/iscsi> cd iqn.2014-06.com.example:remotedisk1/tpg1/
/iscsi/iqn.20...sk1/tpg1> acls/ create iqn.2014-06.com.example:desktop0
Created NODE ACL for iqn.2014-06.com.example:desktop0
/iscsi/iqn.20...:server0/tgp1> ls
o- tpg1 ................................................. [no-gen-acls, no-auth]
  o- acls ............................................................ [ACLs: 1]
  | o- iqn.2014-06.com.example:desktop0 ....................... [Mapped LUNs: 0]
  o- luns ............................................................ [LUNs: 0]
  o- portals ...................................................... [Portals: 0]

ACL configures target to accept initiator connections from client presenting iqn.2014-06.com.example:desktop0 as initiator IQN
- Also known as initiator name

iSCSI Target Configuration

Step 6

In TPG, create LUN for each backstore:

Also activates each backstore

Because ACLs exist for TPG, they are automatically assigned to each LUN created

/iscsi/iqn.20...:server0/tgp1> luns/ create /backstores/block/block1
Created LUN 0.
Created LUN 0->0 mapping in node ACL iqn.2014-06.com.example:desktop0
/iscsi/iqn.20...:server0/tgp1> luns/ create /backstores/block/block2
Created LUN 1.
Created LUN 1->1 mapping in node ACL iqn.2014-06.com.example:desktop0
/iscsi/iqn.20...:server0/tgp1> luns/ create /backstores/fileio/file1
Created LUN 2.
Created LUN 2->2 mapping in node ACL iqn.2014-06.com.example:desktop0
/iscsi/iqn.20...:server0/tgp1> ls
o- tpg1 ................................................. [no-gen-acls, no-auth]
  o- acls ............................................................ [ACLs: 1]
  | o- iqn.2014-06.com.example:desktop0 ....................... [Mapped LUNs: 3]
  |   o- mapped_lun0 .................................. [lun0 block/block1 (rw)]
  |   o- mapped_lun1 .................................. [lun1 block/block2 (rw)]
  |   o- mapped_lun2 .................................. [lun2 fileio/file1 (rw)]
  o- luns ............................................................ [LUNs: 3]
  | o- lun0 ............................ [block/block1 (/dev/iSCSI_vg/disk1_lv)]
  | o- lun1 ......................................... [block/block2 (/dev/vdb2)]
  | o- lun2 .................................. [fileio/file1 (/root/disk1_file)]
  o- portals ...................................................... [Portals: 0]

iSCSI Target Configuration

Step 7

Inside TPG, create portal configuration to designate listening IP address and ports:

Use system’s public network interface
Without specifying TCP port, defaults to standard iSCSI port (3260)

If no IP specified, 0.0.0.0 is used

Permits connections on all network interfaces defined on server

/iscsi/iqn.20...:server0/tgp1> portals/ create 192.168.0.101
Using default IP port 3260
Created network portal 192.168.0.101:3260
/iscsi/iqn.20...:server0/tgp1> ls
o- tpg1 ................................................. [no-gen-acls, no-auth]
  o- acls ............................................................ [ACLs: 1]
  | o- iqn.2014-06.com.example:desktop0 ....................... [Mapped LUNs: 3]
  |   o- mapped_lun0 .................................. [lun0 block/block1 (rw)]
  |   o- mapped_lun1 .................................. [lun1 block/block2 (rw)]
  |   o- mapped_lun2 .................................. [lun2 fileio/file1 (rw)]
  o- luns ............................................................ [LUNs: 3]
  | o- lun0 ............................ [block/block1 (/dev/iSCSI_vg/disk1_lv)]
  | o- lun1 ......................................... [block/block2 (/dev/vdb2)]
  | o- lun2 .................................. [fileio/file1 (/root/disk1_file)]
  o- portals ...................................................... [Portals: 1]
    o- 192.168.0.101:3260 ................................................... [OK]

iSCSI Target Configuration

Step 8

View the entire configuration, then exit targetcli:

targetcli automatically saves upon exit

Configuration file stored in JSON format

/iscsi/iqn.20...:server0/tgp1> cd /
/> ls
o- / ..................................................................... [...]
  o- backstores .......................................................... [...]
  | o- block .............................................. [Storage Objects: 2]
  | | o- block1 ....... [/dev/iSCSI_vg/disk1_lv (100.0MiB) write-thru activated]
  | | o- block2 ...................... [/dev/vdb2 (1.0GiB) write-thru activated]
  | o- fileio ............................................. [Storage Objects: 1]
  | | o- file1 .............. [/root/disk1_file (100.0MiB) write-back activated]
  | o- pscsi .............................................. [Storage Objects: 0]
  | o- ramdisk ............................................ [Storage Objects: 0]
  o- iscsi ........................................................ [Targets: 1]
  | o- iqn.2014-06.com.example:remotedisk1 ........................... [TPGs: 1]
  |   o- tpg1 ........................................... [no-gen-acls, no-auth]
  |     o- acls ...................................................... [ACLs: 1]
  |     | o- iqn.2014-06.com.example:desktop0 ................. [Mapped LUNs: 3]
  |     |   o- mapped_lun0 ............................ [lun0 block/block1 (rw)]
  |     |   o- mapped_lun1 ............................ [lun1 block/block2 (rw)]
  |     |   o- mapped_lun2 ............................ [lun2 fileio/file1 (rw)]
  |     o- luns ...................................................... [LUNs: 3]
  |     | o- lun0 ...................... [block/block1 (/dev/iSCSI_vg/disk1_lv)]
  |     | o- lun1 ................................... [block/block2 (/dev/vdb2)]
  |     | o- lun2 ............................ [fileio/file1 (/root/disk1_file)]
  |     o- portals ................................................ [Portals: 1]
  |       o- 192.168.0.101:3260 ............................................. [OK]
  o- loopback ..................................................... [Targets: 0]
/> exit
Global pref auto_save_on_exit=true
Last 10 configs saved in /etc/target/backup.
Configuration saved to /etc/target/saveconfig.json

iSCSI Target Configuration

Steps 9-10

Add port exemption to default firewall for port 3260:

[root@server0 ~]# firewall-cmd --add-port=3260/tcp
[root@server0 ~]# firewall-cmd --add-port=3260/tcp --permanent

Enable target.service systemd unit:
- target.service recreates target configuration from JSON file at boot
- If step skipped, configured targets work until machine is rebooted
- After reboot, server offers no targets
  [root@server0 ~]# systemctl enable target

Authentication

Can implement ACL node verification and password-based authentication
Can require authentication during iSCSI discovery phase
- Can be unidirectional or bidirectional
CHAP authentication does not use strong encryption for passing of credentials
- Offers additional factor of authentication
- Should not be considered secure
To secure iSCSI data, controlling network side of protocol is better method
- Providing dedicated network or VLANs to pass iSCSI traffic is more secure implementation of protocol

Command-Line Mode

Demonstration ran targetcli in interactive mode

Can also use targetcli to execute commands via command line

Example: Use targetcli to create backstore device, IQN, and activate portal
- Does not present usable target
- Demonstrates several actions executed with targetcli
Can use this method to script target configuration

saveconfig executed at end of command list

In command-line mode, must use saveconfig`to save `targetcli configuration as JSON

[root@server0 ~]# targetcli /backstores/block create block1 /dev/vdb
Created block storage object block1 using /dev/vdb.
[root@server0 ~]# targetcli /iscsi create iqn.2014-06.com.example:remotedisk1
Created target iqn.2014-06.com.example:remotedisk1.
Created TPG 1.
[root@server0 ~]# targetcli /iscsi/iqn.2014-06.com.example:remotedisk1/tpg1/portals create 192.168.0.101
Using default IP port 3260
Created network portal 192.168.0.101:3260.
[root@server0 ~]# targetcli saveconfig
Last 10 configs saved in /etc/target/backup.
Configuration saved to /etc/target/saveconfig.json

References

targetcli(8) man pages
LINUX I/O Targetcli

iSCSI Initiator Introduction

In Red Hat Enterprise Linux, iSCSI initiator typically implemented in software
- Functions similar to hardware iSCSI host bus adapter (HBA)
- Accesses targets from remote storage server
Software-based iSCSI initiator requires connecting to Ethernet network of sufficient bandwidth to carry expected storage traffic
Can also implement iSCSI using hardware initiator
- Includes required protocols in dedicated HBA
HBAs and TCP offload engines (TOEs) move the processing iSCSI, TCP overhead, Ethernet interrupts to hardware
- Eases load on system CPUs
- TOE includes TCP network stack on Ethernet NIC

iSCSI Initiator Introduction

Configuring Initiator

Configuring iSCSI client initiator requires iscsi-initiator-utils
Includes:
- iscsi and iscsid services
- /etc/iscsi/iscsid.conf and /etc/iscsi/initiatorname.iscsi configuration files
Client requires unique IQN
- /etc/iscsi/initiatorname.iscsi contains generated IQN using Red Hat’s domain
- Typically reset tIQN to own domain and appropriate client system string
/etc/iscsi/iscsid.conf contains default settings for node records created during target discovery
Settings include:
- Timeouts
- Retry parameters
- Authentication usernames and passwords

To change file, restart iscsi:

[root@desktop1~]# systemctl restart iscsi

iSCSI Initiator Introduction

Discovering Targets

To discover targets:
- Install iscsi-initiator-utils
- Enable and start iscsi service
Must discover targets before device connection and use
Discovery process stores target node information settings in /var/lib/iscsi/nodes
- Uses defaults from /etc/iscsi/iscsid.conf
Same target can exist on multiple portals
- Node records stored for each portal

To perform discovery:

[root@desktop1~]# iscsiadm -m discovery -t sendtargets -p target_server[:port]
192.168.0.101:3260,1 iqn.2014-06.com.example:server1.example.com

iSCSI Initiator Introduction

In discovery mode, sendtargets request returns targets with access for initiator
Can omit port number if target server configured on port 3260
Upon discovery, node record written to /var/lib/iscsi/nodes and used for subsequent logins
To use the listed target:
```
[root@desktop1~]# iscsiadm -m node -T iqn.2014-06.com.example:server1.example.com [-p target_server[:port]] -l
```
- Specifying portal optional
- If target exists on multiple portals, performing login without specifying portal connects to every node that accepts target name

iSCSI Initiator Introduction

Obtaining Target Information

To obtain information about targets, use iscsiadm
To set command detail levels, use -P N
- 0 specifies least verbose output
Examples:
- iscsiadm -m discovery [-P 0|1]: Show information about discovered targets
- iscsiadm -m node [-P 0|1]: Show information about known targets
- iscsiadm -m session [-P 0|1|2|3]: Show information about active sessions

iSCSI Initiator Introduction

Discontinuing Targets

To discontinue using target, use iscsiadm
- Logs out temporarily
- Node records remain after logout
  - Used to automatically log in to targets on system reboot or iscsi restart
To log out of a target:
```
[root@desktop1~]# iscsiadm -m node -T iqn.2012-04.com.example:example [-p target_server[:port]] -u
```
- If no portal specified, target logs out of all relevant portals
  - Discovery not necessary to log in to target again

iSCSI Initiator Introduction

To permanently log out of target, must delete node records
- Manual/automatic login cannot reoccur without performing discovery
- Not specifying portal removes target node records for all relevant portals

To delete node record permanently:

[root@desktop1~]# iscsiadm -m node -T iqn.2012-04.com.example:example [-p target_server[:port]] -o delete

References

iscsiadm(8) and iscsid(8) man pages
Open-iSCSI

Summary

Introduction to iSCSI
iSCSI Fundamentals
iSCSI Target Overview
iSCSI Target Configuration
Authentication
Command-Line Mode
iSCSI Initiator Introduction

Module Completion

Nice job!

Click the button below to complete this module of the course: